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Abstract. Let F q be the finite field with q elements. Consider the standard 
embedding GL n (F q ) <^-> GL n+ i(F q ). In this paper we prove that for every 
irreducible representation tt of GL n+ i(F q ) over algebraically closed fields of 
characteristic different from 2 we have dim,-K GLn ^ Fq ^ < 2. 

To do that we define a property of weak Gelfand pair and prove a generaliza- 
tion of Gelfand trick for weak Gelfand pairs, using the anti-involution transpose 
to get the result for GL n +i(F q ),GL n (F q ). In a similar manner we show that 
for q not a power of 2 (O n +i(F q ), O n (F q )) is a Gelfand pair over algebraically 
closed fields of characteristic different from 2. 

1 Introduction 

1.1 Gelfand pairs 

A Gelfand pair is a pair of a group and its sub-group, usually denoted by G > H , 
such that if n is an irreducible representation of G then the dimension of the 
space of H invariant vectors in 7r is at most 1. 

The theory was developed in the setting of Lie groups in a paper by I. M. 
Gelfand [2], and today has various applications. Finite and infinite Gelfand pairs 
have been studied in asymptotic and geometric group theory in connection with 
branch groups introduced by R.I. Grigorchuk. Other applications were associ- 
ation schemes of coding theory, orthogonal polynomials and special functions. 
Gelfand pairs are also used in the study of finite Markov chains. For more uses 
and applications see [1]. 

1.2 Structure of the proof 

The pair GL(n+l),GL(n) has been studied over various fields, and was proven 
to be a Gelfand pair (better yet - a strong Gelfand pair) over local fields (see 
[3], [4], [5], [6], [7]). Over finite fields however it is not true, although we can 
still say something about the pair. For some q = p n for p prime, look at the 
groups GL n+ i(F q ) D GL n (F q ). We want to be able to quantify "how much" 
they are not a Gelfand pair. One way to do that is using the following lemma. 
Note that throughout the paper we will only consider finite groups. 



Lemma 1.1. Gelfand trick. Let H < G be groups. Suppose we have an anti- 
involution a : G — > G that preserves all H double- co sets. Then over algebraically 
closed fields of characteristic zero (G, H) is a Gelfand pair. 

Looking at the lemma one might wonder what we can say if we have an 
anti-involution that preserves almost all double-cosets - we still want to have a 
property similar to being a Gelfand pair, but weaker. We also want the result 
to be valid over more fields. We get such a property using Gelfand-Kazhdan 
trick. Denoting by F some algebraically closed field of characteristic different 
from 2, we acquire the following lemma. 

Lemma 1.2. Let H < G be groups, tt irreducible representation of G over F , 
(ir*) H 7^ 0, a anti-involution which preserves H and the central character of it, 
and the number of elements of H\ G/Z(G)H which are not preserved under a 
is 2k, then dimir H < k + 1. 

In some cases the assumption (n*) H ^ can be removed using the following 
lemma. 

Lemma 1.3. If the character of F does not divide \G\, or if there exists an anti- 
involution a of G which preserves H and conjugacy classes, Then dim(n H ) = 
dim{{TT*) H ). 

Finally, we use the last two lemmas for GL n +i(F q ),GL n (F q ) using the nat- 
ural anti-involution transpose, and getting the k from the following lemma. 

Lemma 1.4. For n > 1, the number of GL n (F q ) double-cosets in GL n+ i{F q ) 
(up to the center of GL n+ i(F q )) which are not preserved under transpose is 2. 

Corollary 1.5. For every ir irreducible representation of GL n+ i{F q ) over F 
we have dim(ir GLn( - F i'>) < 2. 

For the case F = C it can be derived from [8J. 

A similar result can be obtained for the orthogonal case. 

Theorem 1.6. For q not a power of 2, (O n +i{F q ),O n (F q )) is a Gelfand pair 
over F. 
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2 Weak Gelfand Pair Property 

We want to see some kind of "weak" Gelfand Pair property. 
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Lemma 2.1. Let G be a group, tt irreducible representation of G over F , a 
anti-involution, then for every v\, V2 S tt, ^ ip G tt* if the matrix coefficients 
(p(ir(g)vi),ip(ir(g)v2) He in F(G) a then vi,v 2 are linearly dependent. 

Proof. Define £^(011,0:2) = Lp(n(ai)Tr(a2)vi) = ip{ir{ai * a 2 )vi) = if{ir{o-{ai * 
oi2))vi) — (p(ir(o~(a2))'K(o~(ai))vi). Since tt is irreducible, if Vi =/= then ■n(a 2 )Vi 
ranges over all tt, and so the left kernels of E>i are equal. From the right hand 
side of the last equation, the right kernels are equal too, i.e. v\,V2 are in the 
kernels of the exact same linear functionals and so Vi are linearly dependent 
(since for every two independent vectors there is a linear functional that sends 
one to zero and the other not). □ 

Lemma 2.2. Let H < G be groups, tt irreducible representation of G over F, 
(tt*) h 7^ ; a anti-involution which preserves H and the central character of tt, 
and the number of elements of H \ G/Z{G)H which are not preserved under a 
is 2k, then dirmr H < k + 1. 

Proof. Assume dirmr 11 > k + 2. Note that by Schur's lemma the center of 
G acts by scalars on tt. Let us denote the central character by Z n , and by 
F(G) HxHx(z( - G) ' Z ^ the space of H x H invariant functions in F(G) such that 
multiplication by an element of Z(G) multiplies the result by the corresponding 
scalar in Z n . Using the assumption that the characteristic of F is not 2, we can 
write this space in terms of even and odd function: {^F{G) HxHyl ^ z ^ G ^ Z7 '^)' 7 ® 
(F(G) HxHx(z( - G) > z * ) ) a - . Note that the the space of odd functions is of dimen- 
sion k. Now take ^ if e {n*) H and k + 2 independent vectors v\, V2, ■ ■ ■ , € 
n H . They create elements of F{G) HxHx ^ zlyG ^ z ^ by ip(-K(g)vi), and by lemma 
2.1 they are linearly independent in F{G) HxHx( ^ z ^ G ^ Z7 '\ and so its dimension 
is at least k + 2. So we can take two linearly independent vectors 1*1,1*2 as a 
linear combination of the Vi such that (p(n(g)ui) are in (F(G) H xHx (. z ( G )-Z w )y ^ 
But then from the previous lemma we have linear dependency of ui,u 2 - con- 
tradiction. □ 

Finally, we want to show that the assumption (tt*) h ^ is not needed in 
some cases, with the following lemma. 

Lemma 2.3. If the character of F does not divide \G\ then dim(ir H ) — dim({Ti* ) H ) . 

Proof. Look at 7r,7r* as representations of H. Notice that a preserved vector 
under H corresponds to the identity representation as a sub-representation. In 
that sense, we get: 

dim{-K H ) =< tt, 1 >,dim((TT*) H ) =< n* , 1 > 

and since 1 = 1* and for every tt, t representations we have < ir, r >=< r* , tt* > 
we get dim(TT H ) = dim((TT*) H ) (using symmetry of intertwining number over 
F). □ 

This lemma shows us the assumption (tt*) h ^ is not needed in specific 
cases, but we still want to give an easier constraint than (tt*) h ^ for the 
general case. We can do that by the following lemmas. 
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Lemma 2.4. Let a ^ be an anti-involution of G which preserves conjugacy 
classes. Define a representation of G on the space of tt by Tr'(g) = Tr{a(g))~ l . 
Then tt* = tt' . 

Proof. We will show that the modular characters of tt* , tt' are equal, and hence 
(from a theorem by R. Brauer) they are isomorphic, tt* can be defined as 
(Tr(gY)' 1 acting on the space of tt. Hence we need to show that the modular 
characters of 7r(g)', Tr(a(gj) are equal, which means that they have the same 
eigenvalues. Since a preserves conjugacy classes, there is some a € G such that 
a(g) = aga~ l , which gives us 

^("(5)) = ^(aga^ 1 ) = 7r(a)7r(5)7r(a _1 ) = 7r(a)7r(.g)7r(a) _1 

which means that 7r(<?), Tr(a(g)) are conjugate, and hence have the same 
eigenvalues. Finally, we know transpose preserves eigenvalues and we get tt* = 
tt'. □ 

If a also preserves H we get the wanted result. 

Lemma 2.5. Let a be an anti-involution of G which preserves H and conjugacy 
classes. Then dim(TT H ) — dim{{-K*) H ). 

Proof. From lemma 3.1 it's enough to show that dim{iT H ) = dim((Tr') H ) which 
is obvious because for a vector to be preserved under H, it needs to be preserved 
under 

TriaiH))- 1 = tt(H)- 1 = TT{H- r ) = tt(H) 

□ 

Notice that if we take a — a then the only assumption we need to add to 
lemma 2.2 instead of (tt*) h ^ is that a preserves conjugacy classes. 

3 Application To GL(n+l),GL(n) Over Finite Fields 

To use lemma 2.2 for GL n+ i{F q ),GL n {F q ) we need to calculate k. 

Lemma 3.1. For n > 1, the number of GL n (F q ) double-cosets in GL n+ i{F q ) 
(up to the center of GL n+ i(F q )) which are not preserved under transpose is 2 
(i.e. k = l). 

Proof. A general double coset is of the form 

(B 0\ (C 0\ 

where B G GL n (F q ), C G GL n (F q ) and A G GL n+1 (F q ). We will write 
A = ^ ^j, where A G M n (F q ), v,ip are vectors in F 7 q l and A is a scalar. 
Because A is in GL n+ i(F q ). it is easy to see that rank(A) is either norn-1. 
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Let us assume rank(A) — n (alternatively n — 1). In such a case, we know that 
using row and column operation we can bring A to the form /„ (7„_i). Note 
that we can apply such operations using B and C. therefore, we can look only 
at the case where A — I n (7„_i). Note that then, after matrix multiplication, 
the upper left n x n matrix is BC (£?/„_iC). Because it needs to be of the form 
of A t , we get BC = I n (BI n - X C = I n -i), i.e. C = B~ l . Finally let us write 
down the multiplication explicitly: 

(B 0\ //„ v\ (B- 1 0\ _ (B Bv\ (B- 1 0\ _ ( I n Bv\ 
{0 l) \<p X) \ l) \<p x)\Q l) {pB- 1 X) 

(alternatively 

We need to verify that there is such B so that it equals A 1 . So we get the 
following equalities: 

Bv = ip* ipB^ 1 = v l 

multiply the left one by B and take transpose, we get B*v — if 1 . First 
assume ip,v ^ 0, and so it's enough to find a symmetric matrix B £ GL n {F q ) 
such that Bv = ip*. This will be proved in the next lemma. So now assume ip 
or v is (Therefore the case of rank n — 1 is dismissed, because then we'll have 
a row/column of zeros). It's easy to see that in this case there is no sufficient 
matrix B, and so we are left with checking how many such double-cosets exist. 
Assuming v — and ipi ^ from the above calculation we see that all such 
matrix are in the same double-coset, because it is enough to find an invertible 
matrix B such that (piB^ 1 = ip 2 which is obvious. So up to the center of 
GL n+ i(F q ) (i.e. scalar multiplication, which allows us to assume A = 1,0, but 
since A = means ip ^ 0, v ^ we may assume A = 1) we get 2 double-cosets 
{<p> = 0, v ^ 0, <p> ^ 0, v = 0), which are transpose of each other. □ 

Lemma 3.2. Over F q , for every n-vectors ip, v ^ there exists a symmetric 
matrix B 6 GL n (F q ) such that B<p> = v. 

Proof. We use induction, Using the following form: 

GO ~ ( r i A ) CO 

Were oi, £>i, c are scalars, v\,r\,(pi are (n-l)-vectors and A is an (n-l)-square 
matrix. Let's first solve the trivial case - if oi, b\, v±, ipi ^ then set c = (Z1/&1, 
r\ = and acquire A by using induction on vi = Aip\. If ip\ = 0, then since 
^^Owe get bi ^ 0. So obviously c = ai/bi,ri — All we need now is 

a suitable A, with no restraints other than to be symmetric, and so that the 
whole matrix B will be invertible. By row and column swaps we can assume 
that the non-zero elements of r\ are the upper-most. If n = then a\ ^ and 




•5 



we can take A = I n -\. So assume n ^ and we choose the following A (we 
show here the whole B): 

[rih l r ih ••• 
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1 



[ri]i 
[rih 
[ri] 3 
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By row and column operations, 
it to the form: 
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Which is invertible, and hence B is invertible. Since B~ Y is symmetric iff 
B is symmetric, the case ip\ = solves the case v\ = 0, and all is left is the 
case a\ = or b\ = 0, which are again the same. So assume b\ = 0. By 
induction there's a symmetric invertible A such that v\ = Aip\. Now we need 
to find suitable n , c. We choose arbitrary n - with the only constrain that 
< n, <pi >= oi, which is easy, for c we have two options: or 1. If both don't 
work, it means that the rows 2 to n have the vector ei in their span. But if 
we look at the vectors from their second place, they are vectors in A which are 
linearly independent, and so all the coefficients are 0, and we get a contradiction. 
Hence or 1 is always good, and that concludes the proof. □ 

Combining lemmas 2.2, 2.5 and 3.1 (for the case a = a = transpose) we 
finally get the result for G = GL n+1 (F q ), H = GL n (F q ). 

Corollary 3.3. For every ir irreducible representation of GL n +\{F q ) over F 
we have dim{-K GL ^ F ^) < 2. 



4 0(n+l),0(n) Over Finite Fields 

Theorem 4.1. For q not a power of 2, (O n+ i(F q ),O n (F q )) is a Gelfand pair 
over F. 
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Proof. We use lemmas 2.2, 2.5 with a = a = transpose. Denote G = 0(n + 
l),H = 0(n). Since H\G/H G\(G/HxG/H) (which is true for every groups 
G, if), and since G/if is isomorphic to the unit sphere, it's enough to show that 
for any unit vectors u, v there is an element g G G such that gu = v,gv = u 
(since transpose translates to (u, v) n> (v, u)). Ifu — v is not orthogonal to itself, 
take g(x) = x — 2 < /^"L v v / x X > ( u — v ) (i- e - the reflection relative to the hyperplane 
orthogonal to u — v). 

<u-v,u> 1- <u,v > 
g (u) = u-2- -(u-v)=u-2- — — -(u-v)=v 

<u — v,u — v> 2 — 2<u,v> 

<u-v,v> <u,v>-l 

q(v) — v — 2 (u — v = v — 2 (u — v) = u 

yy ' <u-v,u-v> y ' 2-2<u,v> y ' 

If u — v is orthogonal to itself, notice that then 

=< u — v,u — v >= 2 — 2 < u, v >^>< u, v >= 1 
and therefore 

<u + v,u + v >=2 + 2 <u,v >=4j^0 

so we can take g(x) = 2 <" +t, ^> (u + v)-x= <H+^£> (u + v)- x (i.e. the 
reflection relative to u + v). 

, . < u + v,u > , . . . 

g(u) = (u + v) — u = [u + v) — u = v 

< u + v,v > , . . . 

g(v) = {u + v) — v = (u + v) — v = u 

□ 
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